1. Technical Field
The present invention relates in general to a method and system for enhancing the usability of data processing systems by visually impaired users, and more particularly to a method and system for permitting visually impaired users to utilize a graphical user interface.
2. Description of the Related Art
In recent years, there has been a move among computer application software developers toward graphical user interfaces. In a graphical user interface, objects, such as file cabinets, folders, documents, and printers, are displayed on the computer screen as miniature graphic representations or icons. These objects are comprised of a group of pixels on the computer screen which may be selected to resemble physical object analogous to the displayed object's function in the computer environment. These objects should not be confused with "object code" or "objects" as utilized in "object-based" or "object-oriented" programming languages. In the graphic user interface, users may manipulate these objects in ways which are similar to the manner in which such analogous objects are manipulated in the work place. Users may manipulate these objects with a mouse or other graphical pointing device to perform desired computer operations.
For example, in the work place, in order to file a document in a folder which is located within a file cabinet, a user will open the file cabinet, locate and open the correct folder, and then place the document inside. Similarly, in the "electronic work place" of the graphical user interface, the user performs the process on a computer screen. Thus, a user will open the file cabinet icon, locate the correct folder icon, and then drop the document icon in the located folder. Users are thus able to transfer their knowledge of a real work place in order to perform similar operations in the computer.
Normally sighted individuals find graphical user interfaces intuitive and easy to work with. However, except for an occasional "beep" or similar tone, graphical user interfaces are virtually silent and the vast majority of the information which such interfaces provide to the user is visual. For example, the appearance of the object or icon may convey information to the user about the data the object represents or how the object interacts with other objects. An object representing an application may have an appearance related to the function of the application, such as, for example, a drawing application may be .represented by an object or icon that resembles a T square and a pencil. Similarly, documents created by an application will be represented by an object that resembles the object that represents the parent application, thereby conveying information about the format of the document.
Other objects may receive and contain objects, such as, for example, an object that resembles a file folder may be utilized to contain other objects representing applications, and documents created by such applications, as an aid to filing and organizing data within the disk drive. Still other objects may perform a function within the data processing system, such as, for example, an object that resembles a trash can may be utilized to perform the function of deleting selected files from a disk drive.
When operating a computer utilizing a graphical user interface, it may be helpful for the user to mentally classify objects into classes of objects, such as, for example, device, data, and container classes. By utilizing this classification, a user, who may be looking for a particular data file, is able to search each container object for the particular data file because the user is able to distinguish container objects, which contain other objects from data objects or device objects, which do not contain other objects. Thus, if a user is able to recognize and distinguish classes of objects, the user may be able to find a desired data file, or complete other similar computer operations, in a more efficient manner.
Since visually impaired users are not able to visually recognize and distinguish classes of objects displayed on the computer screen, graphical user interfaces are essentially not usable by severely visually impaired people. Moreover, even if a visually impaired person were able to locate objects within a graphical user interface, such a user may not be able to benefit from the large amount of information that would be communicated graphically if the user was sighted.
Although visually impaired computer users currently benefit from many forms of adaptive technology, including speech synthesis, large-print processing, braille desk top publishing, and voice recognition, almost none of the foregoing tools have been adapted for use with a graphical user interface. However, there have been a few suggestions of how to incorporate such adaptive technology for use with a graphical user interface. For example, an article published in Byte Magazine suggests that programmers could write software with built-in voice labels for icons. Lazzaro, Windows of Vulnerability, Byte Magazine, (June 1991), page 416. In another example, various synthetic or recorded speech solutions for making computer display contents available to visually impaired persons have been suggested in Golding et al., IBM Technical Disclosure Bulletin, Vol. 26, No. 10B, pages 5633-5636 (March 1984); and Barnett et al., IBM Technical Disclosure Bulletin, Vol. 26, No. 10A, pages 4950-4951 (March 1984). In each of these examples, a visually impaired user is not aided in locating objects on the computer display. Nor do these examples suggest how a user may determine what kind of objects are available on the computer display screen.
Additionally, systems have been suggested which include a mouse with a braille transducer so that a blind mouse user may read text and obtain certain tactile position feedback from such a mouse. Comerford, IBM Technical Disclosure Bulletin No. 3, Vol. 28, page 1343 (August 1985); and Affinito, et al., IBM Technical Disclosure Bulletin No. 12, Vol. 31, page 386 (May 1989). However, while such systems announce various text items, either audibly or by means of a braille transducer in the mouse, and may provide some information to a visually impaired user, such systems do not enable a user to navigate about, and locate objects on, the computer display screen. Nor do such systems enable a user to determine how many disk drives, for example, are connected to the computer system.
It has also been suggested that an audible cursor positioning and pixel (picture element) status identification mechanism may be utilized to help a user of an interactive computer graphics system locate data by utilizing aural feedback to enhance visual feedback. As the cursor in such a system is stepped across the screen, an audible click is generated which varies in tone, corresponding in tone to the current status of each pixel encountered. With this combination of audible and visual cursor feedback, it becomes a more simple task to identify a desired line by noting the change in tone as the cursor moves. For color display applications, each color is represented by a distinct tone so that any single pixel may be distinguished from surrounding pixels of a different color. It has been suggested that this system is especially helpful for visually impaired or learning disabled users. Drumm et al., IBM Technical Disclosure Bulletin No. 48, Vol. 27, page 25-28 (September 1984). However, the foregoing disclosure does not suggest a means of enabling a visually impaired user to navigate about, or locate objects within, a graphical user interface on a computer display screen, nor does it suggest a means of enabling such a user to distinguish objects among multiple classes of objects.
Recently, in a patent application entitled "Method and System for Enabling a Blind Computer User to Handle Message Boxes in a Graphical User Interface," U.S. patent application Ser. No. 07/746,838, filed Aug. 19, 1991, a system has been proposed which permits a visually impaired user to interact with a so-called "message box" within a graphical user interface. As those skilled in the art will appreciate, each message box consists of an icon, explanatory text, and one or more "push buttons." The icon allows the user to identify visually the type of message. The text typically explains the situation and may provide assistance. The textual content may be a question or a statement. Push buttons provided within a message box typically allow the user to interact with the message box.
This proposed system permits visually impaired users to accommodate a message box by announcing the textual contents of such a box when the message box first appears. Thereafter, the push buttons available to respond to the message box are also announced in order from left to right. A homing signal is then provided for finding the message box which increases in pitch as the mouse pointer approaches the message box. When the pointer enters the message box, the message box text and available push buttons are reannounced and the pointer is automatically moved to a default push button. By utilizing this system, a visually impaired user may locate a message box within a computer display system; however, this system fails to provide any suggestion of a manner in which a visually impaired user may selectively locate and distinguish graphical user interface objects that belong to one class from those objects belonging to another class within a graphical user interface.
Another method and system, which have also been recently proposed in a patent application entitled "Audio User Interface with Stereo and Filtered Sound Effects," U.S. patent application Ser. No. 07/746,840, filed Aug. 19, 1991, permits a visually impaired user to locate a mouse pointer or other graphical pointing device within the client area of a window within a graphical user interface by providing a stereo sound system and varying the intensity of the left and right audio channels to indicate a position of the mouse pointer. This system also proposes an increase in pitch of an associated sound to indicate the position of the pointer in the top or bottom of the client area of a window. While this system permits a visually impaired user to manipulate a mouse pointer within a graphical user interface, it fails to show or suggest any technique whereby a particular one of a group of displayed graphical objects may be selected by such a user, or whereby a user may distinguish objects of one class form those of another.
Yet another method and system have been recently proposed in a patent application entitled "Method and System for Enabling Blind or Visually Impaired Computer Users to Graphically Select Displayed Elements," U.S. patent application Ser. No. 07/802,956, filed Dec. 5, 1991, wherein graphic elements within a computer display may be located by a visually impaired user by associating a selected radius defining an area of interest surrounding the vertex of a pointer, controlled by a mouse or other graphical pointing device, and thereafter generating composite audible signals which include identifiable audible signals associated with each graphic element within the computer display. By generating a composite signal, or by cyclicly generating a composite signal, having elements of each identifiable audible signal associated with a graphic object having picture elements within a rotatable sector of the area of interest, a visually impaired user may rapidly and efficiently locate selected objects within the computer display. While this method and system permits a visually impaired user to locate graphical user interface objects within a computer display, it fails to show or suggest any technique whereby a user may distinguish objects of one class from those of another.